Multifunctional integrated image sensor and application to virtual interface technology

ABSTRACT

An electronic camera including an electronic imaging sensor providing an output representing an imaged field, a first imaging functionality employing the electronic imaging sensor for taking a picture of a scene in the imaged field, a second imaging functionality employing the electronic imaging sensor for data entry responsive to user hand activity and a user-operated imaging functionality selection switch operative to enable a user to select operation in one of the first and second imaging functionalities.

REFERENCE TO CO-PENDING APPLICATIONS

This application claims priority from the following now abandoned U.S.patent applications:

U.S. Provisional application Ser. No. 60/392,376, entitled “VirtualInterface Implementation and Application”, filed Jun. 26, 2002 and U.S.Provisional application Ser. No. 60/438,327, entitled “MultifunctionalIntegrated Image Sensor and Application to Virtual InterfaceTechnology”, filed Jan. 7, 2003.

BACKGROUND OF THE INVENTION

The following patents and publications are believed to represent thecurrent state of the art:

U.S. Pat. Nos. 6,377,238; 6,281,238; 5,767,842; 5,457,550; 5,181,181 and4,843,568.

Foreign Patent Documents: WO 00/21024; EP 0982 676 A1; DE 298 02 435 U1;WO 01/93182 and WO 02/054169.

SUMMARY OF THE INVENTION

The present invention relates to data input methods and apparatusgenerally.

There is thus provided in accordance with a preferred embodiment of thepresent invention an electronic camera including an electronic imagingsensor providing an output representing an imaged field, a first imagingfunctionality employing the electronic imaging sensor for taking apicture of a scene in the imaged field, a second imaging functionalityemploying the electronic imaging sensor for data entry responsive touser hand activity and a user-operated imaging functionality selectionswitch operative to enable a user to select operation in one of thefirst and second imaging functionalities.

There is also provided in accordance with another preferred embodimentof the present invention a portable telephone including telephonefunctionality, an electronic imaging sensor providing an outputrepresenting an imaged field, a first imaging functionality employingthe electronic imaging sensor for taking a picture of a scene in theimaged field, a second imaging functionality employing the electronicimaging sensor for data entry responsive to user hand activity and auser-operated imaging functionality selection switch operative to enablea user to select operation in one of the first and second imagingfunctionalities.

There is further provided in accordance with yet another preferredembodiment of the present invention a digital personal assistantincluding at least one personal digital assistant functionality, anelectronic imaging sensor providing an output representing an imagedfield, a first imaging functionality employing the electronic imagingsensor for taking a picture of a scene in the imaged field, a secondimaging functionality employing the electronic imaging sensor for dataentry responsive to user hand activity and a user-operated imagingfunctionality selection switch operative to enable a user to selectoperation in one of the first and second imaging functionalities.

In accordance with another preferred embodiment of the present inventionthe electronic imaging sensor includes a CMOS sensor.

Preferably, the first imaging functionality incorporates a first lensarrangement upstream of the electronic imaging sensor. Additionally, thesecond imaging functionality incorporates a second lens arrangementupstream of the electronic imaging sensor.

In accordance with yet another preferred embodiment of the presentinvention the second imaging functionality incorporates an infraredlight generator for defining a light beam, impingement of whichresulting from the user hand activity produces a light patternrepresentative of the data. Additionally, the second imagingfunctionality also incorporates at least one infrared passing lightfilter associated with the electronic imaging sensor.

In accordance with still another preferred embodiment of the presentinvention the second imaging functionality incorporates a processorreceiving an output from the electronic imaging sensor for providing adata entry output. Additionally, the processor is incorporated togetherwith the electronic imaging sensor on a single chip.

In accordance with another preferred embodiment of the present inventionthe first functionality is a color photography functionality.Additionally or alternatively, the second functionality is an infraredbeam impingement sensing data entry functionality.

In accordance with yet another preferred embodiment of the present theuser-operated imaging functionality selection switch is a mechanicallyresponsive switch which selectably mechanically associates the first andsecond lens arrangements upstream of the electronic imaging sensor.Alternatively or additionally, the user-operated imaging functionalityselection switch is a mechanically responsive switch which selectablymechanically associates selected filters with the electronic imagingsensor.

In accordance with still another preferred embodiment of the presentinvention the user-operated imaging functionality selection switch is anelectronically responsive switch which selectably electronicallydetermines responsiveness to selected outputs from predeterminedportions of the electronic imaging sensor.

In accordance with another preferred embodiment of the present inventionthe electronic camera also includes an array of red, green, blue andinfrared passing pixelwise elements arranged over the electronic imagingsensor. Additionally or alternatively, the electronic camera alsoincludes a data entry template projector employed by the second imagingfunctionality. Preferably, the electronic camera also includes anilluminator employed by the second imaging functionality. Additionallyor alternatively, the electronic camera also includes a synchronizedillumination power variation functionality.

In accordance with another preferred embodiment of the present inventionthe portable telephone also includes an array of red, green, blue andinfrared passing pixelwise elements arranged over the electronic imagingsensor. Additionally or alternatively, the portable telephone alsoincludes a data entry template projector employed by the second imagingfunctionality. Preferably, the portable telephone also includes anilluminator employed by the second imaging functionality. Additionallyor alternatively, the portable telephone also includes a synchronizedillumination power variation functionality.

In accordance with another preferred embodiment of the present inventionthe digital personal assistant also includes an array of red, green,blue and infrared passing pixelwise elements arranged over theelectronic imaging sensor. Additionally or alternatively, the digitalpersonal assistant also includes a data entry template projectoremployed by the second imaging functionality. Preferably, the digitalpersonal assistant also includes an illuminator employed by the secondimaging functionality. Additionally or alternatively, the digitalpersonal assistant also includes a synchronized illumination powervariation functionality.

There is also provided in accordance with another preferred embodimentof the present invention an electronic camera including an electronicimaging sensor providing an output representing an imaged field and anarray of red, green, blue and infrared passing pixelwise elementsarranged over the electronic imaging sensor.

There is further provided in accordance with still another preferredembodiment of the present invention a portable telephone includingtelephone functionality, an electronic imaging sensor providing anoutput representing an imaged field and an array of red, green, blue andinfrared passing pixelwise elements arranged over the electronic imagingsensor.

There is also provided in accordance with another preferred embodimentof the present invention a digital personal assistant including at leastone personal digital assistant functionality, an electronic imagingsensor providing an output representing an imaged field and an array ofred, green, blue and infrared passing pixelwise elements arranged overthe electronic imaging sensor.

There is even further provided in accordance with yet another preferredembodiment of the present invention an electronic camera including anelectronic imaging sensor providing an output representing an imagedfield, a first imaging functionality employing the electronic imagingsensor in a visible radiation imaging mode for taking a picture of ascene in the imaged field and a second imaging functionality employingthe electronic imaging sensor in an infrared radiation sensing mode fordata entry responsive to user hand activity.

There is also provided in accordance with another preferred embodimentof the present invention a portable telephone including telephonefunctionality, an electronic imaging sensor providing an outputrepresenting an imaged field, a first imaging functionality employingthe electronic imaging sensor in a visible radiation imaging mode fortaking a picture of a scene in the imaged field and a second imagingfunctionality employing the electronic imaging sensor in an infraredradiation sensing mode for data entry responsive to user hand activity.

There is further provided in accordance with yet another preferredembodiment of the present invention a digital personal assistantincluding at least one personal digital assistant functionality, anelectronic imaging sensor providing an output representing an imagedfield, a first imaging functionality employing the electronic imagingsensor in a visible radiation imaging mode for taking a picture of ascene in the imaged field and a second imaging functionality employingthe electronic imaging sensor in an infrared radiation sensing mode fordata entry responsive to user hand activity.

In accordance with another preferred embodiment of the present inventionthe electronic camera includes an array of red, green, blue and infraredpassing pixelwise elements arranged over the electronic imaging sensor.

In accordance with yet another preferred embodiment of the presentinvention the portable telephone includes an array of red, green, blueand infrared passing pixelwise elements arranged over the electronicimaging sensor.

In accordance with yet another preferred embodiment of the presentinvention the digital personal assistant includes an array of red,green, blue and infrared passing pixelwise elements arranged over theelectronic imaging sensor.

Preferably, in the first imaging functionality output of the pixelsunderlying the red, green and blue passing elements are employed.Additionally, output of the pixels underlying the infrared passingelements is employed to prevent distortion of color response of theimaging sensor. Additionally, in the second imaging functionality outputof the pixels underlying the infrared passing elements are employed.

In accordance with yet another preferred embodiment of the presentinvention the electronic camera also includes a processor. In accordancewith still another preferred embodiment of the present invention theelectronic camera forms part of a telephone. Alternatively, theelectronic camera forms part of a personal digital assistant.Alternatively, the electronic camera forms part of a wristwatch.

In accordance with another preferred embodiment of the present inventionthe portable telephone also includes a processor.

In accordance with yet another preferred embodiment of the presentinvention the digital personal assistant also includes a processor.

In accordance with still another preferred embodiment of the presentinvention the processor includes the following functionality: as eachpixel value is acquired, determining, using pixel coordinates, whetherthat pixel lies within a predefined keystroke region, acquiring pixelvalues for the pixel coordinates lying within the predefined keystrokeregion, adding or subtracting each of the pixel values to or from apixel total maintained for each the keystroke region based ondetermining a pixel function of each pixel, comparing the pixel totalfor each the keystroke region with a current key actuation threshold, ifthe pixel total exceeds the key actuation threshold for a givenkeystroke region in a given frame and in the previous frame the pixeltotal did not exceed the key actuation threshold for that keystrokeregion, providing a key actuation output and if the pixel total does notexceed the key actuation threshold for a given keystroke region in agiven frame and in the previous frame the pixel total did exceed the keyactuation threshold for that keystroke region, providing a keydeactuation output.

In accordance with another preferred embodiment of the present inventionthe determining whether that pixel lies within a predefined keystrokeregion is made by employing a pixel index table which indicates for eachpixel, whether that pixel lies within a predetermined keystroke regionand, if so, within which keystroke region it lies. Alternatively, bothof the determining steps employ the pixel index table.

In accordance with yet another preferred embodiment of the presentinvention the pixel total is maintained for each keystroke region in akeystroke region accumulator table.

In accordance with still another preferred embodiment of the presentinvention the comparing employs a keystroke region threshold table.

In accordance with another preferred embodiment of the present inventionthe processor also includes the following functionality: once all of thepixels in a frame have been processed, determining an updated backgroundlevel for a frame and determining a key actuation threshold for thekeystroke region threshold table by subtracting the updated backgroundlevel from a predetermined threshold level which is established for eachkeystroke region. Additionally, the pixel function includes adding thepixel values of a plurality of pixels in the keystroke region.

In accordance with yet another preferred embodiment of the presentinvention the processor is operative to determine the “center ofgravity” of pixel values of pixels in the two-dimensional image sensor.Preferably, the processor includes the following functionality: as eachpixel value is acquired, determining, using the pixel coordinates,whether that pixel lies within a predefined active region, acquiringpixel values for various pixel coordinates and determining the “centerof gravity” of the pixel values.

In accordance with still another preferred embodiment of the presentinvention determining the “center of gravity” is achieved by multiplyingthe pixel values by X and Y values representing the geographic positionof each pixel, summing the results along mutually perpendicular axes Xand Y, summing the total of the pixel values for all relevant pixels forthe active region and dividing the summed results by the total of thepixel values to determine the X and Y coordinates of the “center ofgravity”, which represents a desired engagement location.

Preferably, the pixel values are thresholded prior to summing thereof.

There is yet further provided in accordance with another preferredembodiment of the present invention a wristwatch including wristwatchfunctionality, an electronic imaging sensor providing an outputrepresenting an imaged field, a first imaging functionality employingthe electronic imaging sensor for taking a picture of a scene in theimaged field, a second imaging functionality employing the electronicimaging sensor for data entry responsive to user hand activity and auser-operated imaging functionality selection switch operative to enablea user to select operation in one of the first and second imagingfunctionalities.

There is also provided in accordance with another preferred embodimentof the present invention a vehicle including a frame, a drive train anda chassis, including at least one lockable entry portal and portalunlocking functionality and an access control assembly for governingauthorized access to the at least one lockable entry portal including atleast one at least partially light transmissive user finger engagementsurface accessible from outside the chassis, an illuminator operative toilluminate at least one user finger engagement plane adjacent the atleast one user finger engagement surface, a two-dimensional imagingsensor viewing the at least one user finger engagement surface, from alocation inwardly of the at least one finger engagement surface, forsensing light from the illuminator scattered by engagement of a user'sfinger with the at least one finger engagement surface, and a data entryprocessor receiving an output from the two-dimensional imaging sensorand providing a data entry input to the portal unlocking functionality.

There is further provided in accordance with yet another preferredembodiment of the present invention access control apparatus for usewith a vehicle including a chassis having at least one lockable entryportal and portal unlocking functionality and also including at leastone at least partially light transmissive user finger engagement surfaceaccessible from outside the chassis, the access control apparatusincluding an illuminator operative to illuminate the at least one userfinger engagement surface, a two-dimensional imaging sensor viewing theat least one user finger engagement surface from a location inwardly ofthe at least one finger engagement surface for sensing light from theilluminator scattered by engagement of a user's finger with the at leastone finger engagement surface and a data entry processor receiving anoutput from the two-dimensional imaging sensor and providing a dataentry input to the portal unlocking functionality.

There is further provided in accordance with yet another preferredembodiment of the present invention access control apparatus for usewith an enclosure having at least one lockable entry portal and portalunlocking functionality and also including at least one at leastpartially light transmissive user finger engagement surface accessiblefrom outside the enclosure, the access control apparatus including anilluminator operative to illuminate the at least one user fingerengagement surface, a two-dimensional imaging sensor viewing the atleast one user finger engagement surface from a location inwardly of theat least one finger engagement surface for sensing light from theilluminator scattered by engagement of a user's finger with the at leastone finger engagement surface and a data entry processor receiving anoutput from the two-dimensional imaging sensor and providing a dataentry input to the portal unlocking functionality.

There is yet further provided in accordance with still another preferredembodiment of the present invention data entry apparatus for use with atleast one at least partially light transmissive user finger engagementsurface, the data entry apparatus including an illuminator operative toilluminate the at least one user finger engagement surface, atwo-dimensional imaging sensor viewing the at least one user fingerengagement surface from a location inwardly of the at least one fingerengagement surface for sensing light from the illuminator scattered byengagement of a user's finger with the at least one finger engagementsurface and a data entry processor receiving an output from thetwo-dimensional imaging sensor and providing a data entry input.

Preferably, the illuminator is located at a location inwardly of the atleast one engagement surface. Additionally or alternatively, the atleast one finger engagement surface includes a one-dimensional array ofa plurality of finger engagement surfaces. Alternatively, the at leastone finger engagement surface includes a two-dimensional array of aplurality of finger engagement surfaces.

In accordance with another preferred embodiment of the present inventionthe vehicle also includes an illumination director cooperating with theilluminator for providing an illumination beam generally parallel to theat least one finger engagement surface.

In accordance with yet another preferred embodiment of the presentinvention the access control apparatus also includes an illuminationdirector cooperating with the illuminator for providing an illuminationbeam generally parallel to the at least one finger engagement surface.

In accordance with still another preferred embodiment of the presentinvention the data entry apparatus also includes an illuminationdirector cooperating with the illuminator for providing an illuminationbeam generally parallel to the at least one finger engagement surface.

Preferably, the illumination director includes a prism.

There is also provided in accordance with another preferred embodimentof the present invention data entry apparatus for use with at least oneengagement surface, the data entry apparatus including a light emittingdata entry engagement element, which emits light only when it is in atleast predetermined propinquity to the at least one engagement surface,a two-dimensional imaging sensor viewing the at least one engagementsurface for sensing light emitted by the light emitting data entryengagement element and a data entry processor receiving an output fromthe two-dimensional imaging sensor and providing a data entry input.

Preferably, the light emitting data entry engagement element includes anelongated element having an LED at an end thereof. Additionally, the LEDincludes an IR emitting LED. Additionally or alternatively, the dataentry apparatus also includes a proximity switch.

There is further provided in accordance with yet another preferredembodiment of the present invention data entry apparatus for use with atleast one engagement surface, the data entry apparatus including aprojector operative to illuminate the at least one engagement surfaceand to define thereon at least one touchpad region and at least onekeyboard region, the touchpad region being defined by a zero'th orderdiffracted image, an illuminator operative to illuminate the at leastone engagement surface, a two-dimensional imaging sensor viewing the atleast one engagement surface from a location inwardly of the at leastone engagement surface for sensing light from the illuminator scatteredby engagement of a user's finger with the at least one engagementsurface and a data entry processor receiving an output from thetwo-dimensional imaging sensor and providing a data entry input.

Preferably, the processor includes the following functionality: as eachpixel value is acquired, determining, using pixel coordinates, whetherthat pixel lies within a predefined keystroke region, acquiring pixelvalues for the pixel coordinates lying within the predefined keystrokeregion, adding or subtracting each of the pixel values to or from apixel total maintained for each the keystroke region based ondetermining a pixel function of each pixel, comparing the pixel totalfor each the keystroke region with a current key actuation threshold, ifthe pixel total exceeds the key actuation threshold for a givenkeystroke region in a given frame and in the previous frame the pixeltotal did not exceed the key actuation threshold for that keystrokeregion, providing a key actuation output and if the pixel total does notexceed the key actuation threshold for a given keystroke region in agiven frame and in the previous frame the pixel total did exceed the keyactuation threshold for that keystroke region, providing a keydeactuation output.

There is even further provided in accordance with still anotherpreferred embodiment of the present invention a data input deviceincluding an illuminator operative to illuminate at least one engagementplane by directing light along the at least one engagement plane, atwo-dimensional imaging sensor, including an optical wedge element,viewing the at least one engagement plane from a location outside the atleast one engagement plane for sensing light from the illuminatorscattered by engagement of a data entry object with the at least oneengagement plane and a data entry processor receiving an output from thetwo-dimensional imaging sensor and providing a data entry input toutilization circuitry.

Preferably, the optical wedge element enables the imaging sensor toefficiently sense light from the engagement plane.

There is still further provided in accordance with yet another preferredembodiment of the present invention a data input device including anilluminator operative to illuminate at least one engagement plane bydirecting light along the at least one engagement plane, atwo-dimensional imaging sensor, including a pair of oppositely orientedoptical wedge elements, viewing the at least one engagement plane from alocation outside the at least one engagement plane for sensing lightfrom the illuminator scattered by engagement of a data entry object withthe at least one engagement plane and a data entry processor receivingan output from the two-dimensional imaging sensor and providing a dataentry input to utilization circuitry.

There is also provided in accordance with still another preferredembodiment of the present invention a data input device including anilluminator operative to illuminate at least one engagement plane bydirecting light along the at least one engagement plane and toilluminate a proper positioning confirmation location on an engagementsurface, a two-dimensional imaging sensor, including, viewing the atleast one engagement plane from a location outside the at least oneengagement plane for sensing light from the illuminator scattered byengagement of a data entry object with the at least one engagement planeand a data entry processor receiving an output from the two-dimensionalimaging sensor and providing a data entry input to utilizationcircuitry.

Preferably, the imaging sensor senses the presence or absence ofillumination at the proper positioning confirmation location.Additionally, the data entry apparatus is operative to unpower itselfwhen the imaging sensor senses the absence of illumination at the properpositioning confirmation location.

There is further provided in accordance with yet another preferredembodiment of the present invention gaming apparatus including at leastone game piece, a game template projector, operative to project a gametemplate onto a surface, an illuminator operative to illuminate thesurface and an imaging sensor viewing the surface for sensing light fromthe illuminator reflected by at least one optically encoded reflectormounted on the at least one game piece.

In accordance with another preferred embodiment of the present inventioneach of the at least one reflector identifies one of the at least onegame piece and is operative to indicate a location of the at least onegame piece on the game template. Additionally or alternatively, the atleast one reflector also identifies a spatial orientation of the gamepiece.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully fromthe following detailed description, taken in conjunction with thedrawings in which:

FIGS. 1A & 1B are simplified pictorial illustrations of an electroniccamera constructed and operative in accordance with a preferredembodiment of the present invention in two different modes of operation;

FIGS. 2A & 2B are simplified pictorial illustrations of acamera-equipped cellular telephone, constructed and operative inaccordance with a preferred embodiment of the present invention in twodifferent modes of operation;

FIGS. 3A & 3B are simplified pictorial illustrations of acamera-equipped personal digital assistant, constructed and operative inaccordance with a preferred embodiment of the present invention in twodifferent modes of operation;

FIGS. 4A & 4B are simplified diagrammatic illustrations of the operationof imaging functionality selection in accordance with a preferredembodiment of the present invention in two different modes of operation;

FIGS. 5A & 5B are simplified diagrammatic illustrations of the operationof imaging functionality selection in accordance with another preferredembodiment of the present invention in two different modes of operation;

FIG. 6 is a simplified diagrammatic illustration of a data entry objectengagement location sensing subsystem useful in the embodiments of anyof FIGS. 1A-5B;

FIG. 7 is a simplified flow chart illustrating operation of the dataentry object engagement location sensing subsystem of FIG. 6 inaccordance with a preferred embodiment of the present invention;

FIG. 8 is a simplified illustration of a preferred data structureemployed in the operation of the data entry object engagement locationsensing subsystem shown in FIG. 7;

FIG. 9 is a simplified pictorial illustration of outlines of typicalkeystroke regions as sensed by a two-dimensional image sensor viewing akeyboard, such as the keyboard seen in FIG. 6;

FIG. 10 is a simplified pictorial illustration of outlines of typicalfootprints of a typical light pattern occasioned by data entry objectengagement with several keystroke regions, such as those shown in FIG.9;

FIGS. 11A, 11B and 11C are simplified illustrations of three alternativemethodologies for determining the function of the pixel within thekeystroke region in which it lies as shown in FIG. 7;

FIGS. 12A, 12B and 12C are simplified illustrations of traces which areuseful in understanding FIGS. 11A, 11B and 11C;

FIG. 13 is a simplified flow chart illustrating operation of a dataentry object engagement location sensing subsystem employed in atracking system and methodology constructed and operative in accordancewith a preferred embodiment of the present invention;

FIG. 14 is a simplified illustration showing synchronized illuminationpower variation functionality useful in accordance with a preferredembodiment of the present invention;

FIG. 15 is a simplified partially pictorial, partially diagrammaticillustration of data entry apparatus embodied in a vehicle accesscontrol system;

FIG. 16 is a simplified partially pictorial, partially diagrammaticillustration of a variation in the data entry apparatus of FIG. 15;

FIG. 17 is a simplified pictorial illustration of data entry apparatusconstructed and operative in accordance with another preferredembodiment of the present invention;

FIGS. 18A and 18B are respectively a simplified pictorial and asimplified sectional illustration of data entry apparatus constructedand operative in accordance with another preferred embodiment of thepresent invention;

FIGS. 19A and 19B are respectively a simplified pictorial and asimplified sectional illustration of data entry apparatus constructedand operative in accordance with yet another preferred embodiment of thepresent invention;

FIGS. 20A and 20B are respectively a simplified pictorial and asimplified sectional illustration of data entry apparatus constructedand operative in accordance with still another preferred embodiment ofthe present invention;

FIG. 21 is a simplified pictorial illustration of data entry apparatusconstructed and operative in accordance with a further preferredembodiment of the present invention;

FIG. 22 is a simplified pictorial illustration of data entry apparatusconstructed and operative in accordance with another preferredembodiment of the present invention;

FIG. 23 is a simplified pictorial illustration of gaming apparatusconstructed and operative in accordance with a preferred embodiment ofthe present invention; and

FIG. 24 is a simplified pictorial illustration of gaming apparatusconstructed and operative in accordance with another preferredembodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Reference is now made to FIGS. 1A & 1B, which are simplified pictorialillustrations of an electronic camera constructed and operative inaccordance with a preferred embodiment of the present invention in twodifferent modes of operation. As seen in FIGS. 1A & 1B, an electroniccamera 100, which may be of conventional construction other than asspecifically described hereinbelow, such as a Nikon COOLPIX 5700,available from Nikon Corporation, Tokyo, Japan, is equipped with anelectronic imaging sensor module 102, which may include a CMOS array,for example, CMOS array OVT6130, available from Omnivision Technologies,Inc., 1341 Orleans Drive, Sunnyvale, Calif., USA, providing an outputrepresenting an imaged field.

A user-operated imaging functionality selection switch 104 is preferablyprovided to enable a user to select operation in one of two possibleimaging functionalities. In a first imaging functionality, illustratedin FIG. 1A, the electronic imaging sensor module 102 is employed to takea picture of a scene in the imaged field. In a second imagingfunctionality, illustrated in FIG. 1B, the electronic imaging sensormodule 102 is employed for data entry responsive to user hand activity.

Various embodiments of the first and second imaging functionalities willbe described hereinbelow, with reference to FIGS. 4A-5B. Preferably, butnot necessarily, the second imaging functionality, as shown in FIG. 1B,employs a data entry template projector 106, which projects a data entrytemplate, such as that designated by reference numeral 108, onto asurface, such as a table surface 110. An illuminator 111 preferably isprovided and is operative to illuminate a region adjacent the data entrytemplate 108. The imaging sensor module 102 views the data entrytemplate region for sensing light from the illuminator 111. The secondimaging functionality also preferably incorporates processingfunctionality, an example of which is described in applicant's PublishedPCT Application WO 02/054169 A2, the disclosure of which is herebyincorporated by reference.

It is seen in FIG. 1A that when the electronic camera 100 is operativein the first imaging functionality, also termed a picture mode, adigital picture 112 is provided. Similarly, it is seen in FIG. 1B thatwhen the electronic camera 100 is operative in the second imagingfunctionality, also termed a data mode, a stream of data 114 isprovided.

Reference is now made to FIGS. 2A & 2B, which are simplified pictorialillustrations of a camera-equipped cellular telephone, constructed andoperative in accordance with a preferred embodiment of the presentinvention in two different modes of operation. As seen in FIGS. 2A & 2B,a camera-equipped cellular telephone 200, which may be of conventionalconstruction other than as specifically described hereinbelow, such as aSamsung SPH-I330, available from Samsung Electronics, Seoul, Korea, isequipped with an electronic imaging sensor module 202, which may includea CMOS array, for example, CMOS array OVT6130, available from OmnivisionTechnologies, Inc., 1341 Orleans Drive, Sunnyvale, Calif., USA,providing an output representing an imaged field.

A user-operated imaging functionality selection switch 204 is preferablyprovided to enable a user to select operation in one of two possibleimaging functionalities. In a first imaging functionality, illustratedin FIG. 2A, the electronic imaging sensor module 202 is employed to takea picture of a scene in the imaged field. In a second imagingfunctionality, illustrated in FIG. 2B, the electronic imaging sensormodule 202 is employed for data entry responsive to user hand activity.

Various embodiments of the first and second imaging functionalities willbe described hereinbelow, with reference to FIGS. 4A-5B. Preferably, butnot necessarily, the second imaging functionality, as shown in FIG. 2B,employs a data entry template projector 206, which projects a data entrytemplate, such as that designated by reference numeral 208, onto asurface, such as a table surface 210. An illuminator 211 preferably isprovided and is operative to illuminate a region adjacent the data entrytemplate 208. The imaging sensor module 202 views the data entrytemplate region for sensing light from the illuminator 211. The secondimaging functionality also preferably incorporates processingfunctionality an example of which is described in applicant's PublishedPCT Application WO 02/054169 A2, the disclosure of which is herebyincorporated by reference.

It is seen in FIG. 2A that when the camera-equipped cellular telephone200 is operative in the first imaging functionality, also termed apicture mode, a digital picture 212 is provided. Similarly, it is seenin FIG. 2B that when the camera-equipped cellular telephone 200 isoperative in the second imaging functionality, also termed a data mode,a stream of data 214 is provided.

Reference is now made to FIGS. 3A & 3B, which are simplified pictorialillustrations of a camera-equipped personal digital assistant,constructed and operative in accordance with a preferred embodiment ofthe present invention in two different modes of operation. As seen inFIGS. 3A & 3B, a camera-equipped personal digital assistant 300, whichmay be of conventional construction other than as specifically describedhereinbelow, such as a Samsung SPH-I700, available from SamsungElectronics, Seoul, Korea, is equipped with an electronic imaging sensormodule 302, which may include a CMOS array, for example, CMOS arrayOVT6130, available from Omnivision Technologies, Inc., 1341 OrleansDrive, Sunnyvale, Calif., USA, providing an output representing animaged field.

A user-operated imaging functionality selection switch 304 is preferablyprovided to enable a user to select operation in one of two possibleimaging functionalities. In a first imaging functionality, illustratedin FIG. 3A, the electronic imaging sensor module 302 is employed to takea picture of a scene in the imaged field. In a second imagingfunctionality, illustrated in FIG. 3B, the electronic imaging sensormodule 302 is employed for data entry responsive to user hand activity.

Various embodiments of the first and second imaging functionalities willbe described hereinbelow, with reference to FIGS. 4A-5B. Preferably, butnot necessarily, the second imaging functionality, as shown in FIG. 3B,employs a data entry template projector 306, which projects a data entrytemplate, such as that designated by reference numeral 308, onto asurface, such as a table surface 310. An illuminator 311 preferably isprovided and is operative to illuminate a region adjacent the data entrytemplate 308. The imaging sensor module 302 views the data entrytemplate region for sensing light from the illuminator 311. The secondimaging functionality also preferably incorporates processingfunctionality an example of which is described in applicant's PublishedPCT Application WO 02/054169 A2, the disclosure of which is herebyincorporated by reference.

It is seen in FIG. 3A that when the electronic camera 300 is operativein the first imaging functionality, also termed a picture mode, adigital picture 312 is provided. Similarly, it is seen in FIG. 3B thatwhen the electronic camera 300 is operative in the second imagingfunctionality, also termed a data mode, a stream of data 314 isprovided.

Reference is now made to FIGS. 4A & 4B, which are simplifieddiagrammatic illustrations of the operation of imaging functionalityselection in accordance with a preferred embodiment of the presentinvention in two different modes of operation. As seen in FIGS. 4A & 4B,one implementation of the invention involves the use of a first andsecond selectably positionable optical assemblies which may beselectably placed in front of an electronic imaging sensor module 402.

In the illustrated embodiment, the sensor module 402 comprises atwo-dimensional CMOS detector array 404 which is overlaid by an array ofcolor filters 406, which pass infrared radiation. A lens 408 focuseslight from an imaging field onto array 404.

FIG. 4A shows a first selectably positionable optical assembly 410,preferably comprising an IR blocking filter 412, aligned with lens 408.The imaging functionality of FIG. 4A is suitable for picture taking,since it prevents IR radiation from distorting the color response of thesensor module 402.

FIG. 4B shows a second selectably positionable optical assembly 420,preferably comprising a visible radiation blocking filter 422, an IRband pass filter 424 and a field lens 426, aligned with lens 408. Theimaging functionality of FIG. 4B is suitable for data input, since itallows only a relatively narrow band of IR radiation to reach the sensormodule 402 from a relatively wide field of view.

Reference is now made to FIGS. 5A & 5B, which are simplifieddiagrammatic illustrations of the operation of imaging functionalityselection in accordance with another preferred embodiment of the presentinvention in two different modes of operation. In this embodiment,imaging functionality selection is effected electronically, without theneed for mechanically movable optical elements.

As seen in FIGS. 5A & 5B, this implementation of the invention involvesthe use of an electronic imaging sensor module 502 including a CMOSdetector array 504 overlaid with a repeating array of pixelwiseselective light transmissive filters 506 characterized in that the arrayincludes red (R), green (G), blue (B) and infrared (IR) transmissivefilters. A lens 508 focuses light from an imaging field onto detectorarray 504.

FIG. 5A shows the imaging functionality of the embodiment of FIGS. 5Aand 5B in a picture taking mode, wherein only the outputs of the pixelsunderlying the R, G and B filters are employed and the respective IRcontent of the outputs of each of the pixels underlying the R, G and Bfilters is subtracted therefrom. This color processing prevents IRradiation from distorting the color response of the sensor module 502.

FIG. 5B shows the imaging functionality of the embodiment of FIGS. 5Aand 5B in a data entry mode, wherein only the outputs of the pixelsunderlying the IR filters are employed.

Reference is now made to FIG. 6, which is a simplified and generalizedillustration of a projection keyboard system and methodology which ispreferably incorporated into the embodiments of FIGS. 1A-5B inaccordance with a preferred embodiment of the present invention. Aprojection subsystem 550, preferably including a solid state lightsource (not shown) which illuminates a spatial light modulation element(not shown), defines an image 552 of a keyboard on a projection surface554, preferably an inert surface, such as a desktop.

An illumination subsystem 556, preferably including a solid state lightsource (not shown), directs light in a radially directed illuminationpattern 558, which preferably extends in a plane generally parallel tothe projection surface 554. It is appreciated that the radially directedillumination pattern 558 has a very narrow spread in the directiongenerally perpendicular to the projection surface 554. It is furtherappreciated that the radially directed illumination pattern 558 ispreferably located very close to the projection surface 554.

Impingement of light in the radially directed illumination pattern 558on a data entry object 560, such as a user's finger, a stylus or otherimplement, causes light to be scattered or reflected therefrom. It isappreciated that the light is only scattered or reflected when the dataentry object 560 is in close contact with the keyboard 552 defined onprojection surface 554.

A detection subsystem 562, preferably employing a solid state imagingsensor (not shown) incorporated in a camera in the embodiments of FIGS.1A-5B, receives light reflected or scattered from data entry object 560.The received light is mapped onto the imaging sensor with respect to arepresentation of the keyboard for associating the location of the dataentry object 560 sensed by detection subsystem 562 with a key location563 on the keyboard 552. Thus, the sensed location of data entry object560 indicates which key of the keyboard 552 is being engaged.

Reference is now made to FIG. 7, which is a simplified flow chartillustrating operation of a data entry object engagement locationsensing subsystem of FIG. 6 in accordance with a preferred embodiment ofthe present invention, and to FIG. 8, which is a simplified illustrationof a preferred data structure employed in the operation of the dataentry object engagement location sensing subsystem shown in FIG. 7.

FIG. 8 shows a simplified illustration of a preferred data structureemployed in the operation of the data entry object engagement locationsensing method described hereinbelow with respect to FIG. 7. It isappreciated that the imaging sensor of a camera, such as camera 100(FIG. 1A), is typically comprised of a set of M×N pixels, wherein aparticular group of pixels views a defined region of the engagementplane which preferably overlies the projection surface 110 (FIG. 1A).Thus, it is possible that a particular pixel group, located in the imageplane of the camera 100, may receive scattered light from a data entryobject 560 (FIG. 6) engaging the key location 563 (FIG. 6).

Thus, as the camera 100 views the projection surface 110, each of theM×N pixels in the image plane of the camera 100 may receive light from acorresponding region in the engagement plane in respect of a data entryobject engagement therewith.

As seen in FIG. 7, pixel values, such as gray level values, are acquiredfor various pixel coordinates. As each pixel value is acquired, adetermination is made, using the pixel coordinates, as to whether thatpixel lies within a predefined keystroke region, such as keystrokeregions 600 shown in FIG. 9. This determination is preferably made byemploying a pixel index table 601 which indicates, for each pixel,whether that pixel lies within a predetermined keystroke region, such askeystroke regions 605, 606, 607 and 608 (FIG. 9), and, if so, withinwhich keystroke region it lies.

The function of the pixel within the keystroke region in which it liesis then determined, preferably by employing table 601. This function istypically additive or subtractive, but may alternatively have anotherfunction. Typically, depending on the function, the pixel value is addedto or subtracted from a pixel total maintained for each keystroke regionin a keystroke region accumulator table 602.

Once all of the pixels in a frame have been processed as aforesaid, anupdated background level is determined for the frame and a key actuationthreshold is determined typically by subtracting the updated backgroundlevel from a predetermined threshold level which is established for eachkeystroke region. This is preferably carried out by employing akeystroke region threshold table 604.

The contents of the keystroke region accumulator table 602 for eachkeystroke region preferably are then compared with the current keyactuation threshold. If the contents of the accumulator table 602 exceedthe key actuation threshold for a given key actuation region in a givenframe and in the previous frame the contents of the accumulator table602 did not exceed the key actuation threshold, a key actuation outputis provided.

Similarly, if the contents of the accumulator table 602 do not exceedthe key actuation threshold for a given key actuation region in a givenframe and in the previous frame the contents of the accumulator table602 did exceed the key actuation threshold, a key deactuation output isprovided. In all other cases, no output need be generated.

Reference is now made to FIG. 9, which is a simplified pictorialillustration of outlines of typical keystroke regions 605, 606, 607 and608 as sensed by a two-dimensional image sensor (FIG. 6) viewing akeyboard, such as the keyboard 552, seen in FIG. 6.

Reference is now made to FIG. 10, which is a simplified pictorialillustration of outlines of typical footprints 609, 610, 611 and 612 ofa typical light pattern occasioned by data entry object engagementcorresponding to the keystroke regions 605, 606, 607 and 608 (FIG. 9).

Reference is now made to FIGS. 11A, 11B and 11C, which are simplifiedillustrations of three alternative methodologies for determining thefunction of the pixel within the keystroke region in which it lies asshown in FIG. 8, and to FIGS. 12A, 12B and 12C, which are simplifiedillustrations of traces which are useful in understanding FIGS. 11A, 11Band 11C.

Turning now to FIG. 11A, there is shown arranged along a commonarbitrary axis 618 a simplified keystroke region 620 corresponding to agiven key and containing a plurality of pixels 622. A typical simplifiedfootprint of a typical light pattern occasioned by data entry objectengagement with the given key is indicated by reference numeral 624.FIG. 10 shows outlines of typical footprints 609, 610, 611 and 612,corresponding to keystroke regions designated 605, 606, 607 and 608 inFIG. 9.

A typical background signal pattern is indicated by reference numeral634. Superimposition of the footprint 624 over the background signalpattern 634 is indicated at reference numeral 635. A one dimensionallyselectable overlap of footprint 624 over keystroke region 620 isindicated at reference numeral 636. A one dimensionally selectableoverlap of background signal pattern 634 over keystroke region 620 isindicated at reference numeral 637. A one dimensionally selectableoverlap of superimposition 635 over keystroke region 620 is indicated atreference numeral 638.

FIG. 12A illustrates a bias function 640 corresponding to a crosssection of the keystroke region 620 taken along axis 618, which biasfunction defines keystroke region 620 along axis 618. There is also seena signal function 644 corresponding to a cross section of the footprint624 along axis 618; a background signal function 646 corresponding to across section of the background signal pattern 634 along axis 618 and acombination signal 648 corresponding to a cross section of thesuperimposition 635.

There is also shown in FIG. 12A a one dimensional convolution 650corresponding to one dimensionally selectable overlap 636; a onedimensional convolution 652 corresponding to one dimensionallyselectable overlap 637 and a one dimensional convolution 654corresponding to one dimensionally selectable overlap 638. Convolution650 is shown with a threshold 660; convolution 652 is shown with athreshold 662 and convolution 654 is shown with a threshold 664.

Turning now to FIG. 11B, there is shown arranged along a commonarbitrary axis 670 a simplified keystroke region 680 corresponding to agiven key and containing a plurality of pixels 682. The simplifiedkeystroke region 680 is here shown surrounded by a simplified keystrokeregion border 683. A typical simplified footprint of a typical lightpattern occasioned by data entry object engagement with the given key isindicated by reference numeral 684. A typical background signal patternis indicated by reference numeral 686. Superimposition of the footprint684 over the background signal pattern 686 is indicated at referencenumeral 688. A one dimensionally selectable overlap of footprint 684over keystroke region 680 and border 683 is indicated at referencenumeral 690. A one dimensionally selectable overlap of background signalpattern 686 over keystroke region 680 and border 683 is indicated atreference numeral 692. A one dimensionally selectable overlap ofsuperimposition 688 over keystroke region 680 and border 683 isindicated at reference numeral 694.

FIG. 12B illustrates a bias function 700 corresponding to a crosssection of the keystroke region 680 and of the border 683 taken alongaxis 670, which bias function defines keystroke region 680 along axis670. It is seen that border 683 is assigned a negative value relative tothe value of the keystroke region 680. This value assignment isappreciated to enhance the value of data entry object engagements to theextent that they lie within the keystroke region 680 and to decrease thevalue of such data entry object engagements to the extent that theyextend over the border 683. There is also seen a signal function 704corresponding to a cross section of the footprint 684 along axis 670; abackground signal function 706 corresponding to a cross section of thebackground signal pattern 686 along axis 670 and a combination signal708 corresponding to a cross section of the superimposition 688.

There is also shown in FIG. 12B a one dimensional convolution 720,corresponding to one dimensionally selectable overlap 690; a onedimensional convolution 722, corresponding to one dimensionallyselectable overlap 692 and a one dimensional convolution 724corresponding to one dimensionally selectable overlap 694. Convolution720 is shown with a threshold 726; convolution 722 is shown with athreshold 727 and convolution 724 is shown with a threshold 728.

Turning now to FIG. 11C, there is shown arranged along a commonarbitrary axis 730 a simplified keystroke region 740 corresponding to agiven key and containing a plurality of pixels 741. The simplifiedkeystroke region 740 is here shown surrounded by a simplified keystrokeregion inner border 742 and by a simplified keystroke region outerborder 743. A typical simplified footprint of a typical light patternoccasioned by data entry object engagement with the given key isindicated by reference numeral 744. A typical background signal patternis indicated by reference numeral 746. Superimposition of the footprint744 over the background signal pattern 746 is indicated at referencenumeral 748. A one dimensionally selectable overlap of footprint 744over keystroke region 740 and borders 742 and 743 is indicated atreference numeral 750. A one dimensionally selectable overlap ofbackground signal pattern 746 over keystroke region 740 and borders 742and 743 is indicated at reference numeral 752. A one dimensionallyselectable overlap of superimposition 748 over keystroke region 740 andborders 742 and 743 is indicated at reference numeral 754.

FIG. 12C illustrates a bias function 760 corresponding to a crosssection of the keystroke region 740 and of the borders 742 and 743 takenalong axis 730, which bias function defines keystroke region 740 alongaxis 730. It is seen that border 742 is assigned a zero value and border743 is assigned a negative value relative to the value of the keystrokeregion 740. It is appreciated that these value assignments enhance thevalue of data entry object engagements that to the extent that they liewithin the keystroke region 740 and to decrease the value of such dataentry object engagements to the extent that they extend across theborder 742 and at least into border 743. There is also seen a signalfunction 764 corresponding to a cross section of the footprint 744 alongaxis 730; a background signal function 766 corresponding to a crosssection of the background signal pattern 746 along axis 730 and acombination signal 768 corresponding to a cross section of thesuperimposition 748.

There is also shown in FIG. 12C a one dimensional convolution 780,corresponding to one dimensionally selectable overlap 750; a onedimensional convolution 782, corresponding to one dimensionallyselectable overlap 752 and a one dimensional convolution 784corresponding to one dimensionally selectable overlap 754. Convolution780 is shown with a threshold 786; convolution 782 is shown with athreshold 787 and convolution 784 is shown with a threshold 788.

It may be appreciated from a consideration of convolutions 638, 694 and754 that the dual border arrangement of FIGS. 11C and 12C provides thebest detection of data entry object keystroke engagement, whileminimizing background effects.

Reference is now made to FIG. 13, which is a simplified flow chartillustrating operation of a data entry object engagement locationsensing subsystem employed in a tracking system and methodologyconstructed and operative in accordance with a preferred embodiment ofthe present invention.

As seen in FIG. 13, pixel values, such as gray level values, areacquired for various pixel coordinates. As each pixel value is acquired,a determination may be made, using the pixel coordinates, as to whetherthat pixel lies within a predefined active region. Typically, if thepixel does not lie within a predetermined active region, its value isignored.

The pixel value for each pixel is preferably thresholded and typicallyall pixel values falling below a given threshold are ignored. Theremaining pixel values may be weighted by a selected weightingparameter. In order to determine the “center of gravity” of thethresholded and weighted pixel values, the thresholded and weightedpixel values are multiplied respectively by X and Y values representingthe coordinate position of each pixel and the results are summed alongmutually perpendicular axes X and Y and stored in X and Y accumulators.The total of the thresholded and weighted pixel values for all relevantpixels are also summed and stored in a data accumulator, for the entireactive region.

Once all of the pixels in a frame have been processed as aforesaid, thesummed thresholded and weighted pixel values along the X and Y axesrespectively are divided by the total of the thresholded and weightedpixel values for the entire active region to determine the X and Ycoordinates of the “center of gravity”, which represents a desiredengagement location.

Reference is now made to FIG. 14, which is a simplified illustrationshowing synchronized illumination power variation functionality usefulin accordance with a preferred embodiment of the present invention. Thefunctionality illustrated in FIG. 14 is directed to modulating theamount of illumination provided for data entry object engagementdetection. This modulation is desirable because the intensity of lightimpinging on a data entry object and thus scattered thereby, decreaseswith the distance between an illuminator 830 and a data entry object.Thus, it may be appreciated that the amount of light impinging on a dataentry object 832 is substantially greater than the amount of lightimpinging on a data entry object 834. Furthermore, the amount ofscattered light impinging on a detector 836 decreases with the distancebetween the data entry object and the detector. These two distancedependencies make detection of data entry object engagement difficult.

In order to overcome this difficulty, there is provided in accordancewith a preferred embodiment of the present invention variable intensitydrive electronics 840 which is coupled to both illuminator 830 anddetector 836 and which causes the intensity of light produced by theilluminator 830 to vary, typically in a ramp fashion, in synchronizationto the imaging field location of detector 836.

Thus, it may be seen that when a near portion (A) of the engagementsurface 554 (FIG. 6) is being imaged, typically at the top portion A ofdetector 836, the intensity is at a minimum. When an intermediateportion (B) of the engagement surface 554 is being imaged, typically atthe middle of detector 836, the intensity is at an intermediate leveland when a far portion (C) of the engagement surface 554 is beingimaged, typically at the bottom portion (C) of the detector 836, theintensity is at a maximum.

Variable intensity drive electronics 840 operates preferably byproviding a synchronization output 842 to detector 836 and acorresponding synchronization output 844 to illuminator 830, causing theintensity level to increase in time in synchronization with the locationof a scanned image region in detector 836.

Reference is now made to FIG. 15, which is a simplified partiallypictorial, partially diagrammatic illustration of data entry apparatusembodied in a vehicle access control system. As seen in FIG. 15, avehicle 1500, typically comprising a frame, a drive train and a chassisincluding at least one lockable entry portal or door 1502 and portalunlocking functionality, includes an access control assembly designatedgenerally by reference numeral 1504.

The access control assembly 1504 preferably includes at least one atleast partially light transmissive user finger engagement surfaceaccessible from outside the chassis. In the illustrated embodiment,plural user finger engagement surfaces 1506 are defined on a car doorwindow 1508. An illuminator 1510, preferably located interiorly of theexterior of the vehicle, such as an IR line generator, is operative toilluminate a finger engagement plane 1512 adjacent the user fingerengagement surfaces 1506. A preferred IR line generator is described inapplicant's Published PCT Application WO02/054169 A2, the disclosure ofwhich is hereby incorporated by reference.

A two-dimensional imaging sensor 1514, located interiorly of theexterior of the vehicle, views the finger engagement surfaces 1506, froma location inwardly of the car door window 1508, for sensing lightoriginating from the illuminator 1510, which is scattered by engagementof a user's finger with the finger engagement surfaces 1506. A preferredtwo-dimensional imaging sensor 1514 is described in applicant'sPublished PCT Application WO02/054169 A2, the disclosure of which ishereby incorporated by reference.

A data entry processor 1516 receives an output from the two-dimensionalimaging sensor 1514 and provides a data entry input to a conventionaldoor unlocking functionality (not shown).

Reference is now made to FIG. 16, which is a simplified diagrammaticillustration of a variation in the data entry apparatus of FIG. 15. Asseen in FIG. 16, a vehicle 1600, typically comprising a frame, a drivetrain and a chassis including at least one lockable entry portal or door1602 and portal unlocking functionality, includes an access controlassembly designated generally by reference numeral 1604.

The access control assembly 1604 preferably includes at least one atleast partially light transmissive user finger engagement surfaceaccessible from outside the chassis. In the illustrated embodiment,plural user finger engagement surfaces 1606 are defined on a car doorwindow 1608. In this embodiment, the plural user finger engagementsurfaces 1606 may be arranged in a two-dimensional array. An illuminator1610, preferably located interiorly of the exterior of the vehicle, suchas an IR line generator, is operative to illuminate a finger engagementplane 1612 adjacent the user finger engagement surfaces 1606. Apreferred IR line generator is described in applicant's Published PCTApplication WO02/054169 A2, the disclosure of which is herebyincorporated by reference.

In the embodiment of FIG. 16, the finger engagement planes 1612 areparallel to the finger engagement surfaces 1606, by virtue of provisionof a prism 1613 disposed typically on the outside of the vehicle window1608 along the light path from illuminator 1610.

A two-dimensional imaging sensor 1614, located interiorly of theexterior of the vehicle, views the finger engagement surfaces 1606, froma location inwardly of the car door window 1608, for sensing lightoriginating from the illuminator 1610, which is scattered by engagementof a user's finger with the finger engagement surfaces 1606. A preferredtwo-dimensional imaging sensor 1614 is described in applicant'sPublished PCT Application WO02/054169 A2, the disclosure of which ishereby incorporated by reference.

A data entry processor 1616 receives an output from the two-dimensionalimaging sensor 1614 and provides a data entry input to a conventionaldoor unlocking functionality (not shown).

It is appreciated that portions of the apparatus of FIGS. 15 and 16 mayfind application in other types of access control environments and invarious data entry environments.

Reference is now made to FIG. 17, which is a simplified pictorialillustration of data entry apparatus constructed and operative inaccordance with another preferred embodiment of the present invention.As seen in FIG. 17, a light emitting data entry engagement element,preferably in the form of a stylus 1700, emits light when and only whenit is in at least predetermined propinquity to an engagement surface1702.

A two-dimensional imaging sensor 1704 views the engagement surface 1702for sensing light emitted by the stylus 1700. A data entry processor1706 receives an output from the two-dimensional imaging sensor 1704 andprovides a data entry input.

As seen in FIG. 17, the stylus 1700 preferably comprises an IR emittingLED 1708 located in an IR transmissive tip 1710 thereof. A proximityswitch 1712, such as a contact switch, provides an output indication ofthe proximity of tip 1710 to engagement surface 1702. Circuitry 1714receives an input from proximity switch 1712 and operates the LED 1708accordingly, so as to provide an IR emission whose location indicatesthe location of stylus impingement with engagement surface 1702.

Reference is now made to FIGS. 18A and 18B, which are respectively asimplified pictorial and a simplified sectional illustration of dataentry apparatus constructed and operative in accordance with anotherpreferred embodiment of the present invention. As seen in FIGS. 18A &18B, the data entry apparatus comprises a projector 1800 operative toilluminate at least one engagement surface 1802 and to define thereon atleast one touchpad region 1804 and at least one keyboard region 1806,the touchpad region 1804 being defined by a zero'th order diffractedimage provided by the projector 1800.

Turning particularly to FIG. 18B, it is seen that the projector 1800comprises a light source, such as a diode laser 1810, which outputs viaa collimating lens 1812 onto a first diffractive optical element 1814.Diffractive optical element 1814 diffracts the light impinging thereoninto a primary keyboard image beam 1816 which impinges on at last onekeyboard region 1806 and into a secondary ghost image beam 1818, whichis preferably blocked by a barrier 1819. Residual light passing throughdiffractive optical element 1814 defines a zero'th order beam 1820,which impinges onto a second diffractive optical element 1822.

Diffractive optical element 1822 diffracts the light impinging thereonfrom the zero'th order beam 1820 into a touchpad defining beam 1824which impinges on at least one touchpad region 1804.

Returning to FIG. 18A, it is seen that there is also provided anilluminator 1830, operative to illuminate the finger engagement surface.A two-dimensional imaging sensor 1832 views the engagement surface forsensing light from the illuminator 1830 scattered by engagement of auser's finger with the finger engagement surface. A data entry processor1834 receives an output from the two-dimensional imaging sensor 1832 andprovides a data entry input.

Reference is now made to FIGS. 19A and 19B, which are respectively asimplified pictorial and a simplified sectional illustration of dataentry apparatus constructed and operative in accordance with yet anotherpreferred embodiment of the present invention. As seen in FIGS. 19A &19B, the data entry apparatus comprises a projector 1900 operative toilluminate at least one finger engagement surface 1902 and to definethereon typically at least one keyboard region 1904. An illuminator 1906is operative to illuminate the finger engagement surface 1902.

An imaging sensor module 1908, including a two-dimensional imagingsensor 1910, views the engagement surface for sensing light from theilluminator 1906 scattered by engagement of a user's finger with thefinger engagement surface 1902. A particular feature of the imagingsensor module 1908 is that it includes an optical wedge element 1912which provides a tilt, enabling a generally vertically disposed imagingsensor 1910 to efficiently sense light from a finger engagement surface1902 extending generally perpendicularly thereto.

A data entry processor 1914 receives an output from the two-dimensionalimaging sensor 1910 and provides a data entry input.

Reference is now made to FIGS. 20A and 20B, which are respectively asimplified pictorial and a simplified sectional illustration of dataentry apparatus constructed and operative in accordance with stillanother preferred embodiment of the present invention. As seen in FIGS.20A & 20B, the data entry apparatus comprises a projector 2000 operativeto illuminate at least one finger engagement surface 2002 and to definethereon typically at least one keyboard region 2004. An illuminator 2006is operative to illuminate the finger engagement surface 2002.

An imaging sensor module 2008, including a two-dimensional imagingsensor 2010, views the engagement surface for sensing light from theilluminator 2006 scattered by engagement of a user's finger with thefinger engagement surface 2002. A particular feature of the imagingsensor module 2008 is that it includes a pair of oppositely orientedoptical wedge elements 2012 and 2014, each of which provides a tilt to aportion of the light incoming to sensor 2010, enabling efficient use ofarea of generally rectangular imaging sensor, as seen at referencenumber 2016.

Reference is now made to FIG. 21, which is a simplified pictorialillustration of data entry apparatus constructed and operative inaccordance with yet another preferred embodiment of the presentinvention. As seen in FIG. 21, the data entry apparatus comprises aprojector 2100 operative to illuminate at least one finger engagementsurface 2102 and to define thereon typically at least one keyboardregion 2104. An illuminator 2106 is operative to illuminate the fingerengagement surface 2102.

An imaging sensor module 2108, including a two-dimensional imagingsensor 2110, views the engagement surface for sensing light from theilluminator 2106 scattered by engagement of a user's finger with thefinger engagement surface 2102.

A particular feature of the illuminator 2106 is that defines anauxiliary light path 2112 which impinges on a proper positioningconfirmation location 2114 on the finger engagement surface 2102adjacent and preferably nearer to the illuminator than the at least onekeyboard region 2104. The imaging sensor module 2108 senses the presenceof illumination at location 2114 and in the absence of sensedillumination at that location, typically indicating improper placementof the data entry apparatus, typically unpowers the data entryapparatus.

A data entry processor 2116 receives an output from the two-dimensionalimaging sensor 2110 and provides a data entry input.

Reference is now made to FIG. 22, which is a simplified pictorialillustration of data entry apparatus constructed and operative inaccordance with another preferred embodiment of the present invention inthe form of a wristwatch. As seen in FIG. 22, a wristwatch 2200, whichmay be of conventional construction other than as specifically describedhereinbelow, is equipped with an electronic imaging sensor module 2202,which may include a CMOS array, for example, CMOS array OVT6130,available from Omnivision Technologies, Inc., 1341 Orleans Drive,Sunnyvale, Calif., USA, providing an output representing an imagedfield.

A user-operated imaging functionality selection switch 2204 ispreferably provided to enable a user to select operation in a data entryfunctionality employing the electronic imaging sensor module 2202 fordata entry responsive to user hand activity.

The data entry functionality preferably employs a data entry templateprojector 2206, which projects a data entry template, such as thatdesignated by reference numeral 2208, onto a surface, such as a tablesurface 2210. An illuminator 2212 is operative to illuminate a regionadjacent the data entry template. The imaging sensor module 2202 viewsthe data entry template region for sensing light from the illuminator2212 scattered by engagement of a user's finger with the data entrytemplate.

The data entry functionality also preferably incorporates processingfunctionality, an example of which is described in applicant's PublishedPCT Application WO 02/054169 A2, the disclosure of which is herebyincorporated by reference. The data entry functionality produces astream of data 2214.

Reference is now made to FIG. 23, which is a simplified pictorialillustration of gaming apparatus constructed and operative in accordancewith a preferred embodiment of the present invention. As seen in FIG.23, gaming apparatus, one example of which is a board game, such aschess, is provided with an electronic imaging sensor module 2302, whichmay include a CMOS array, for example, CMOS array OVT6130, availablefrom Omnivision Technologies, Inc., 1341 Orleans Drive, Sunnyvale,Calif., USA, providing an output representing an imaged field.

A user-operated imaging functionality selection switch 2304 ispreferably provided to enable a user to select operation of one ofplural gaming functionalities, such as different board games, employingthe electronic imaging sensor module 2302 for gaming piece locating.

The gaming functionality preferably employs a gaming template projector2306, which projects a game template, such as a chess board, designatedby reference numeral 2308, onto a surface, such as a table surface 2310.An illuminator 2312 is operative to illuminate a region adjacent thegame template 2308. The imaging sensor module 2302 views the gametemplate region for sensing light from the illuminator 2312 reflected byoptically encoded reflectors 2314 mounted on individual gaming pieces2316. The reflectors 2314 preferably individually identify each gamingpiece and indicate its location on the game template 2308.

The gaming functionality also preferably incorporates processingfunctionality, an example of which is described in applicant's PublishedPCT Application WO 02/054169 A2, the disclosure of which is herebyincorporated by reference. The gaming functionality produces a stream ofdata 2316, which may be used for game management and scoring.

Reference is now made to FIG. 24, which is a simplified pictorialillustration of gaming apparatus constructed and operative in accordancewith another preferred embodiment of the present invention. FIG. 24shows gaming pieces, such as dice 2402, which bear variously differentlycoded reflectors 2404 on various surfaces thereof, thereby to enable notonly the identity and location but also the spatial orientation of thegaming pieces to be sensed.

It will be appreciated by persons skilled in the art that the presentinvention is not limited to what has been particularly shown anddescribed hereinabove. Rather the scope of the present inventionincludes both combinations and subcombinations of the various featuresdescribed hereinabove as well as modifications and variations thereof aswould occur to a person of skill in the art upon reading the foregoingspecification and which are not in the prior art.

1. An electronic camera comprising: an electronic imaging sensorproviding an output representing an imaged field; a first imagingfunctionality employing said electronic imaging sensor for taking apicture of a scene in said imaged field; a second imaging functionalityemploying said electronic imaging sensor for data entry responsive touser hand activity; and a user-operated imaging functionality selectionswitch operative to enable a user to select operation in one of saidfirst and second imaging functionalities, said second imagingfunctionality comprising an illuminator providing synchronizedillumination power variation functionality.
 2. An electronic cameraaccording to claim 1 and wherein said electronic imaging sensorcomprises a CMOS sensor.
 3. An electronic camera according to claim 1and wherein said first imaging functionality incorporates a first lensarrangement upstream of said electronic imaging sensor.
 4. An electroniccamera according to claim 1 and wherein said second imagingfunctionality incorporates a second lens arrangement upstream of saidelectronic imaging sensor.
 5. An electronic camera according to claim 1and wherein said second imaging functionality incorporates an infraredlight generator for defining a light beam, impingement of whichresulting from said user hand activity produces a light patternrepresentative of said data.
 6. An electronic camera according to claim5 and wherein said second imaging functionality also incorporates atleast one infrared passing light filter associated with said electronicimaging sensor.
 7. An electronic camera according to claim 1 and whereinsaid second imaging functionality incorporates a processor receiving anoutput from said electronic imaging sensor for providing a data entryoutput.
 8. An electronic camera according to claim 7 and wherein saidprocessor is incorporated together with said electronic imaging sensoron a single chip.
 9. An electronic camera according to claim 1 andwherein said first imaging functionality is a color photographyfunctionality.
 10. An electronic camera according to claim 1 and whereinsaid second imaging functionality is an infrared beam impingementsensing data entry functionality.
 11. An electronic camera according toclaim 1 and wherein said first imaging functionality incorporates afirst lens arrangement upstream of said electronic imaging sensor andwherein said second imaging functionality incorporates a second lensarrangement upstream of said electronic imaging sensor and wherein saiduser-operated imaging functionality selection switch is a mechanicallyresponsive switch which selectably mechanically associates said firstand second lens arrangements upstream of said electronic imaging sensor.12. An electronic camera according to claim 1 and wherein saiduser-operated imaging functionality selection switch is a mechanicallyresponsive switch which selectably mechanically associates selectedfilters with said electronic imaging sensor.
 13. An electronic cameraaccording to claim 1 and wherein said user-operated imagingfunctionality selection switch is an electronically responsive switchwhich selectably electronically determines responsiveness to selectedoutputs from predetermined portions of said electronic imaging sensor.14. An electronic camera according to claim 1 and comprising an array ofred, green, blue and infrared passing pixelwise elements arranged oversaid electronic imaging sensor.
 15. An electronic camera according toclaim 1 and also comprising a data entry template projector employed bysaid second imaging functionality.
 16. An electronic camera according toclaim 1 and wherein said electronic camera forms part of a telephone.17. An electronic camera according to claim 1 and wherein saidelectronic camera forms part of a personal digital assistant.
 18. Anelectronic camera according to claim 1 and wherein said electroniccamera forms part of a wristwatch.
 19. An electronic camera comprising:an electronic imaging sensor providing an output representing an imagedfield; a first imaging functionality employing said electronic imagingsensor in a visible radiation imaging mode for taking a picture of ascene in said imaged field; and a second imaging functionality employingsaid electronic imaging sensor in an infrared radiation sensing mode fordata entry responsive to user hand activity, said second imagingfunctionality comprising an illuminator providing synchronizedillumination power variation functionality.
 20. An electronic cameraaccording to claim 19 and comprising an array of red, green, blue andinfrared passing pixelwise elements arranged over said electronicimaging sensor.
 21. An electronic camera according to claim 20 andwherein in said first imaging functionality output of the pixelsunderlying the red, green and blue passing elements are employed.
 22. Anelectronic camera according to claim 21 and wherein output of the pixelsunderlying said infrared passing elements is employed to preventdistortion of color response of said imaging sensor.
 23. An electroniccamera according to claim 20 and wherein in said second imagingfunctionality output of the pixels underlying the infrared passingelements are employed.
 24. An electronic camera according to claim 20and also comprising a processor.
 25. An electronic camera according toclaim 20 and wherein said electronic camera forms part of a telephone.26. An electronic camera according to claim 20 and wherein saidelectronic camera forms part of a personal digital assistant.
 27. Anelectronic camera according to claim 20 and wherein said electroniccamera forms part of a wristwatch.
 28. A portable telephone comprising:telephone functionality; an electronic imaging sensor providing anoutput representing an imaged field; a first imaging functionalityemploying said electronic imaging sensor for taking a picture of a scenein said imaged field; a second imaging functionality employing saidelectronic imaging sensor for data entry responsive to user handactivity; and a user-operated imaging functionality selection switchoperative to enable a user to select operation in one of said first andsecond imaging functionalities, said second imaging functionalitycomprising an illuminator providing synchronized illumination powervariation functionality.
 29. A portable telephone comprising: telephonefunctionality; an electronic imaging sensor providing an outputrepresenting an imaged field; a first imaging functionality employingsaid electronic imaging sensor in a visible radiation imaging mode fortaking a picture of a scene in said imaged field; a second imagingfunctionality employing said electronic imaging sensor in an infraredradiation sensing mode for data entry responsive to user hand activity;and an array of red, green, blue and infrared passing pixelwise elementsarranged over said electronic imaging sensor, said second imagingfunctionality comprising an illuminator providing synchronizedillumination power variation functionality.
 30. A digital personalassistant comprising: at least one personal digital assistantfunctionality; an electronic imaging sensor providing an outputrepresenting an imaged field; a first imaging functionality employingsaid electronic imaging sensor for taking a picture of a scene in saidimaged field; a second imaging functionality employing said electronicimaging sensor for data entry responsive to user hand activity; and auser-operated imaging functionality selection switch operative to enablea user to select operation in one of said first and second imagingfunctionalities, said second imaging functionality comprising anilluminator providing synchronized illumination power variationfunctionality.
 31. A digital personal assistant comprising: at least onepersonal digital assistant functionality; an electronic imaging sensorproviding an output representing an imaged field; a first imagingfunctionality employing said electronic imaging sensor in a visibleradiation imaging mode for taking a picture of a scene in said imagedfield; a second imaging functionality employing said electronic imagingsensor in an infrared radiation sensing mode for data entry responsiveto user hand activity; and an array of red, green, blue and infraredpassing pixelwise elements arranged over said electronic imaging sensor,said second imaging functionality comprising an illuminator providingsynchronized illumination power variation functionality.